{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Numpy实现深层神经网络\n",
    "\n",
    "在该实验中我们将介绍如何使用Python及Numpy lib库实现深层神经网络模型来识别猫。本小节代码与第三章Pyhton版本代码大体一致，区别在于增加了3层隐藏层并设置不同节点数。\n",
    "\n",
    "** 图片处理 **\n",
    "\n",
    "由于识别猫问题涉及到图片处理指示，这里对计算机如何保存图片做一个简单的介绍。在计算机中，图片被存储为三个独立的矩阵，分别对应图3-6中的红、绿、蓝三个颜色通道，如果图片是64*64像素的，就会有三个64*64大小的矩阵，要把这些像素值放进一个特征向量中，需要定义一个特征向量X，将三个颜色通道中的所有像素值都列出来。如果图片是64*64大小的，那么特征向量X的总纬度就是64*64*3，也就是12288维。这样一个12288维矩阵就是Logistic回归模型的一个训练数据。\n",
    "\n",
    "<img src=\"images/image_to_vector.png\" style=\"width:550px;height:300px;\">\n",
    "\n",
    "## 1 - 引用库\n",
    "\n",
    "首先，载入几个需要用到的库，它们分别是：\n",
    "- numpy：一个python的基本库，用于科学计算\n",
    "- matplotlib.pyplot：用于生成图，在验证模型准确率和展示成本变化趋势时会使用到\n",
    "- h5py：用于处理hdf5文件数据\n",
    "- dnn_app_utils_v2：包含了一些有用的工具函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import h5py\n",
    "import dnn_app_utils_v2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2 - 数据预处理\n",
    "\n",
    "这里简单介绍数据集及其结构。数据集以hdf5文件的形式存储，包含了如下内容：\n",
    "\n",
    "- 训练数据集：包含了m_train个图片的数据集，数据的标签（Label）分为cat（y=1）和non-cat（y=0）两类。\n",
    "- 测试数据集：包含了m_test个图片的数据集，数据的标签（Label）同上。\n",
    "\n",
    "单个图片数据的存储形式为（num_x, num_x, 3），其中num_x表示图片的长或宽（数据集图片的长和宽相同），数字3表示图片的三通道（RGB）。\n",
    "在代码中使用一行代码来读取数据，读者暂不需要了解数据的读取过程，只需调用load_dataset()方法，并存储五个返回值，以便后续的使用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_x_orig, train_y, test_x_orig, test_y, classes = dnn_app_utils_v2.load_data() \n",
    "#数据预处理 \n",
    "train_x_flatten = train_x_orig.reshape(train_x_orig.shape[0], -1).T   # The \"-1\" makes reshape flatten the remaining dimensions\n",
    "test_x_flatten = test_x_orig.reshape(test_x_orig.shape[0], -1).T\n",
    "\n",
    "#归一化\n",
    "train_x = train_x_flatten/255.\n",
    "test_x = test_x_flatten/255."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3 - 建立神经网络模型\n",
    "\n",
    "对比“浅层神经网络”的网络结构，本小节神经网络模型有以下不同：\n",
    "\n",
    "- 增加2层隐藏层，共有3层隐藏层和1层输出层；\n",
    "\n",
    "- 第一层设置20个节点，第二层7个，第三层5个；\n",
    "\n",
    "- 隐藏层激活函数使用Relu激活函数。\n",
    "\n",
    "在dnn_app_utils_v2文件中已包含下列函数，在实现神经网络模型中将直接调用（下列函数在第四章Python代码部分均有实现，根据网络结构不同略有差异）：\n",
    "\n",
    "- 初始化参数\n",
    "\n",
    "initialize_parameters_deep(layer_dims)\n",
    "\n",
    "\n",
    "- 正向传播\n",
    "\n",
    "L_model_forward(X, parameters)\n",
    "\n",
    "\n",
    "- 成本函数\n",
    "\n",
    "compute_cost(AL, Y)\n",
    "\n",
    "\n",
    "- 反向传播\n",
    "\n",
    "L_model_backward(AL, Y, caches)\n",
    "\n",
    "\n",
    "- 参数更新\n",
    "\n",
    "update_parameters(parameters, grads, learning_rate)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#设置神经网络规模，5个数字分别表示从输入层到隐藏层到输出层各层节点数\n",
    "layers_dims = [12288, 20, 7, 5, 1] \n",
    " \n",
    "#定义函数：L层神经网络模型\n",
    "def L_layer_model(X, Y, layers_dims, learning_rate = 0.0075, num_iterations = 3000, print_cost=False):\n",
    "    \"\"\"\n",
    "    L层神经网络正向传播: [加权偏移->RELU激活]*(L-1)->加权激活->SIGMOID激活.\n",
    "    \n",
    "    参数:\n",
    "    X -- 输入值，维度为 (样本个数, 像素num_px * num_px * 3)\n",
    "    Y -- 真实值（0表示不是猫，1表示是猫），维度为 (1, 样本个数)\n",
    "    layers_dims -- 各层节点数\n",
    "    learning_rate -- 学习率\n",
    "    num_iterations -- 训练次数\n",
    "    print_cost -- 参数设置True，则每100次训练打印一次成本函数值\n",
    "    \n",
    "    返回值:\n",
    "    parameters -- 模型训练所得参数，用于预测\n",
    "    \"\"\"\n",
    "\n",
    "    np.random.seed(1)\n",
    "    costs = []                                                                \n",
    "\n",
    "    #参数初始化\n",
    "    parameters = dnn_app_utils_v2.initialize_parameters_deep(layers_dims)\n",
    "    \n",
    "    #训练\n",
    "    for i in range(0, num_iterations):\n",
    "\n",
    "        #正向传播: [加权偏移 -> RELU激活]*(L-1) -> 加权偏移后 -> SIGMOID激活       \n",
    "        AL, caches = dnn_app_utils_v2.L_model_forward(X, parameters)\n",
    "        \n",
    "        #计算成本函数\n",
    "        cost = dnn_app_utils_v2.compute_cost(AL, Y)\n",
    "    \n",
    "        #反向传播\n",
    "        grads = dnn_app_utils_v2.L_model_backward(AL, Y, caches)\n",
    " \n",
    "        #更新参数\n",
    "        parameters = dnn_app_utils_v2.update_parameters(parameters, grads, learning_rate)\n",
    "                \n",
    "        #每100次训练打印一次成本函数\n",
    "        if print_cost and i % 100 == 0:\n",
    "            print (\"Cost after iteration %i: %f\" %(i, cost))\n",
    "        if print_cost and i % 100 == 0:\n",
    "            costs.append(cost)\n",
    "            \n",
    "    #绘制损失函数变化折线图\n",
    "    plt.plot(np.squeeze(costs))\n",
    "    plt.ylabel('cost')\n",
    "    plt.xlabel('iterations (per tens)')\n",
    "    plt.title(\"Learning rate =\" + str(learning_rate))\n",
    "    plt.show()\n",
    "    \n",
    "    return parameters\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面开始训练，训练2500次，观察成本函数变化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cost after iteration 0: 0.771749\n",
      "Cost after iteration 100: 0.672053\n",
      "Cost after iteration 200: 0.648263\n",
      "Cost after iteration 300: 0.611507\n",
      "Cost after iteration 400: 0.567047\n",
      "Cost after iteration 500: 0.540138\n",
      "Cost after iteration 600: 0.527930\n",
      "Cost after iteration 700: 0.465477\n",
      "Cost after iteration 800: 0.369126\n",
      "Cost after iteration 900: 0.391747\n",
      "Cost after iteration 1000: 0.315187\n",
      "Cost after iteration 1100: 0.272700\n",
      "Cost after iteration 1200: 0.237419\n"
     ]
    }
   ],
   "source": [
    "parameters = dnn_app_utils_v2.L_layer_model(train_x, train_y, layers_dims, num_iterations = 2500, print_cost = True)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在训练集和测试集上进行预测，检测模型准确率，训练集和测试集准确率分别为0.986和0.8：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "print(‘Train Accuracy’)\n",
    "pred_train = predict(train_x, train_y, parameters)\n",
    "print(‘Test Accuracy’)\n",
    "pred_test = predict(test_x, test_y, parameters)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从准确率看，相比于第三章的逻辑回归（本章节处理的数据与第三章相同），深层神经网络准确率提高了不少，这是因为深层神经网络提供更多隐藏层，使得神经网络能够拟合更复杂的模型，识别图片准确率更高。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
